The present invention related to methods for enhancing digital color images. More particularly, the method applies a scaling function to vary the strength of RGB dots within an image without exceeding the inherent dynamic range for the image.
If one were to photograph the stain glass windows of a Cathedral, the resulting image would be typically too dark for modern tastes. The beauty of the actual stained glass is lost. Further, a photographer can increase depth of field of an image by reducing the device""s aperture, but light collection is sacrificed and can also result in a dark image. Further, as a still image device is capable of only recording the image using a predetermined aperture and exposure time, images having a large disparity between light and dark can only make a compromise; adjusting to expose the film, or record, based on either the dark areas or the light areas in an attempt to properly reveal the detail therein. Images resulting from any of the above can benefit from image enhancement.
The usual response by the viewer is to wish to brighten darkened areas and thus reveal the details otherwise obscured to the naked eye. Current techniques produce one or two undesirable effects when brightening: either the colors are washed out into near grays (imagine a nearly black and white and badly faded Stain Glass window); or the brighter areas become very distorted in terms of color and some, completely washed out. The distorted areas tend to be the ones of the most interest, such as the details of a face.
Conventional digital image processing has these same problems when a substantial level of brightening is required. The xe2x80x98standardxe2x80x99 enhancement is to brighten the image (fade it out) and then afterwards, amplify the colors. This solution is only an approximate solution and only works on images that aren""t too bad in the first place. The image becomes badly color-distorted if a substantial amount of brightening is used.
In U.S. Pat. No. 5,982,926 to Kuo et al. (xe2x80x9cKuoxe2x80x9d), Kuo suggests that a color image, particularly one originating from video, can be enhanced much more effectively by first transforming the RGB color space to HSV color space. All operations thereafter are performed on the HSV transformed color space. Once in HSV color space, Kuo then isolates and removes the color information (Hue) from the remaining image components (saturation and intensity). Kuo suggests that the components of saturation and intensity can be enhanced without introducing distortion into the color or Hue component. Kuo""s color image is represented by a plurality of pixels in HSV color space. Once transformed, Kuo inverse transforms HSV back to RGB color space, all the while claiming this to be efficient. In the preferred embodiment, Kuo adjusts intensity (V) and saturation (S). In summary, Kuo first transforms RGB to HSV color space, applies two sequential transformation functions to V and S respectively, and finally inverse transforms the altered HSV back to RGB for display.
Respectfully, Applicant asserts that manipulation of the saturation does affect the color and thus the Kuo technique does not result in true color enhancement. Color photos have three degrees of freedom, being R,G and B, where as black and white photos only have one. Any transformation of the three values results in three more values, each of which contains a color component and not merely a single color value (i.e. hue) and two other independent structural components (i.e. saturation and intensity). Hue may be a more extreme sense of color than saturation, but saturation is still a color component. Adjusting a dot""s saturation results in a change in the proportions of RGB in the dot and hence its color. If the saturation is changed, say as part of brightening the image, the result is not the same as if the recording device or camera had obtained the image directly from the real world subject under brighter conditions, or more exposure.
Further, note that Kuo emphasizes and attempts to minimize the computational overhead or expense. Unfortunately, Kuo introduces two RGB-HSV and HSV-RGB transformations in addition to whatever adjustments (preferably two) Kuo makes to the HSV pixel. A transformation from RGB to HSV color space, and back again, involves the use of computation-intensive mathematical functions.
Accordingly, there is demonstrated a need for a computationally efficient process which is capable of maintaining true color for each dot during enhancement of an image.
The present invention addresses these problems by providing a technique where, no matter how much image-brightening is needed or what the nature of that brightening is, the color of all dots in the image are preserved in all circumstances.
The effect of the present invention is to virtually amplify the light captured by the digital image recorder. This means that once the captured image is processed using the present invention, each dot within the processed image is modified to be the same as if the digital image recorder, had used a different light gathering power or procedure for that dot, including simulating the use of a larger aperture or a longer light gathering duration. This modified light gathering procedure may be applied uniformly across the entire processed image or may vary from dot to dot. By ensuring that the dynamic range of the digital recorder is never exceeded, by operating in the primary RGB color space, and by identically treating each of R,G, and B in a color dot, the virtual light amplification process preserves the true color of the original image.
In a preferred embodiment, and remaining in RGB space, a triplet of RGB values is extracted for each dot of the digital image. The maximum of the RGB triplet is determined for each dot. The maximum of all of the dot maximums, or an image maximum, is determined and a scaling function is defined which provides scaling factors for each dot maximum. The scaling factors, or a function defining same, are determined and applied to the values of the dot maximums and most particularly, the image maximum, so that no resulting value exceeds the known dynamic range of the system. The very same scaling factor, applied to a dot maximum, is also applied to each of the remaining R, G or B of that dot so as to maintain the original proportions or ratios between R,G and B, and thereby maintain the true color.
The enhancement is very efficient, only requiring three simple multiplications, by the same scaling factor, to enhance each color dot. Further, and more preferably, by forming a look-up table of scaling factors for the determined dot maximums, then the calculation of the scaling factor is performed only once. Typically the dynamic range is 0xe2x80x94255 (a maximum of 256 different light strengths) and therefore, for a usual image having in the order of 250,000 dots, at least 1000 of them have the same strength and thus the same scaling factor can be applied to an average of 1000 dots, saving that many calculations. Larger images, which are becoming common, will save even more calculations.
Accordingly in a broad aspect of the invention a method is provided for adjusting a digital image without introducing color distortion. The image is formed of a plurality of color dots, each dot having at least three independent values representing the strength of the three primary colors R, G, and B. Each of the RGB values lies between a minimum and a maximum of the dynamic range for the system.
The method comprises the steps of:
determining the dot maximum of the three RGB values in RGB color space;
applying a predetermined scaling factor to each dot maximum, the scaling factor being such that the scaled dot maximum is always less than or equal to the maximum of the dynamic range of the system; and
applying the same predetermined scaling factor used for a dot maximum to each of that dot""s two remaining R,G or B values.
When adjusted in the above method, the adjusted image comprises a plurality of new scaled RGB values for each dot wherein the ratios between R,G and B for a dot remain the same after scaling as they were before scalingxe2x80x94thereby maintaining true color.
Preferably the scaling factors are obtained from a continuous scaling function. The scaling function normalizes at least a portion of the range of the image to a portion of the dynamic range without ever exceeding the maximum of the dynamic range. Due to the lower magnitude of the dynamic range than number of dots in an image, computation efficiency is improved by first establishing a look-up table of scaling factors.
The preferred scaling function is a lazy xe2x80x9cSxe2x80x9d curve form which produces an aesthetically pleasing enhancement of most digital images.
In another preferred embodiment, one can select only a portion of the image, usually an underexposed area, and normalize subset image range to substantially the entire dynamic range of the system for enhancing the detail therein, all without ever exceeding the system""s dynamic range for any dots and while maintaining true color by being mindful of the ratios of R,G, and B for each dot.